59 participants worth of data usable data (63 finished the experiment in total, but 4 have about 0% correct on naming trials. Note that the participant IDs don’t quite match up to the total nunber of participants. The mismatch is because the RAs accidentally skipped a couple of numbers), recruited during the summer and through SONA. We’re had been aiming for 60.

120 items per participant, encountered in lists of 12 items. With three conditions (binocular, cfs, not-studied), there were four items in each condition in each list. Of those four items, three were encountered in a “go” test trial, and one was in a “no-go” test trial. So, 1/4 of test-trials were no-go. This leaves up to 30 items in go trials per condition, per participant (but the actual numer will be different, depending on how many objects were correctly named).

Figures

Naming Accuracy

The following plot shows the proportion of items that participants named correctly. The first plot shows each participant invididually to see which ones didn’t try during the naming condition. These ones were excluded from the group-level plot and all further plots.

Note that in this group-level plot (and all other following), each participants’ average performance is calculated separetely and then averaged together. Error bars reflect the standard error of the mean, averaging across participant averages (note that I had previously been showing 95% CI).

PAS

Here are the PAS ratings provided to the first and second encounter of items in the study phase. Only items studied in the CFS condition are shown, because these are the only trials in which participants were asked to provide a PAS rating.

Most participants gave ratings of 2 or 3 on most CFS trials. A few were better at stopping the trial before being able to identify the object (provided mostly PAS 2 ratings), but I wouldn’t rule out that those participants were simply pressing 2 because they thought that was the ‘correct’ response.

Go/No-go accuracy

The following plot shows proportion correct on the no/no-go trials (whether participants either said an item was appearing, or correctly waited). Raw accuracy is high. I didn’t calculate anything like d’ given that performance was basically at ceiling for all participants, and so d’ would be infinity.

Percentiles

Percentiles are first calculated within participants, then averaged across participants.

Note that because these lines are conditioned on accurately naming the cue item, there are different numbers of trial in each condition.

These cdf plots are shown for each participant

These next plots contain similar information as the ones above, but now conditions are shown as different from Not-Studied.

Note that in these participant-level plots, it’s a little easier to see that this grouping of naming accuracy x condition – combined with excluding trials where participants were incorrect in go-nogo trials – leaves some participants without any data in some of the conditions. For example, participant 5 didn’t name any of the objects in the Not Studied condition, so participant 5’s graph doesn’t have any ‘cue_correct_fct’ difference lines (there were no Not Studied items to use as a baseline).

The above plots have just 4 percentiles, because that was how many were used in a paper Dave sent me. Previously, I had been grouping the data into 5 percentiles. Below are the different plots with 5 percentiles.

## Fitting one lmer() model. [DONE]
## Calculating p-values. [DONE]

For the actual stats, I ran a linear-mixed effects model to test for 1) an interaction between study condition x percentile when object isn’t named, 2) a post-hoc contrast to test for an effect of CFS when object isn’t named, and 3) a test for an effect of CFS when object isn’t named, at the first quantile.

These ended up being as we wanted them. There was no interaction between condition x percentile, but there was an effect of study under CFS (collapsing across percentiles), and that includes an effect os study at the first percentile.

## Mixed Model Anova Table (Type 2 tests, KR-method)
## 
## Model: RT ~ Condition * Percentile * cue_correct_fct + (1 | subject)
## Data: rt_quant0
##                                 Effect        df         F p.value
## 1                            Condition 1, 862.57   7.35 **    .007
## 2                           Percentile 3, 862.12 17.76 ***  <.0001
## 3                      cue_correct_fct 1, 885.77    4.56 *     .03
## 4                 Condition:Percentile 3, 862.12      0.44     .73
## 5            Condition:cue_correct_fct 1, 862.64      0.04     .84
## 6           Percentile:cue_correct_fct 3, 862.12 11.20 ***  <.0001
## 7 Condition:Percentile:cue_correct_fct 3, 862.12      0.50     .68
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1
## cue_correct_fct = 0:
##  Condition     lsmean         SE     df   lower.CL  upper.CL t.ratio
##  Binocular 0.21440089 0.03554553 122.00 0.16413192 0.2646699   6.032
##  CFS       0.14574874 0.03554553 122.00 0.09547977 0.1960177   4.100
##  p.value
##   <.0001
##   0.0004
## 
## cue_correct_fct = 1:
##  Condition     lsmean         SE     df   lower.CL  upper.CL t.ratio
##  Binocular 0.15763001 0.03697874 139.22 0.10533417 0.2099258   4.263
##  CFS       0.09865568 0.03716827 141.74 0.04609181 0.1512196   2.654
##  p.value
##   0.0002
##   0.0321
## 
## Results are averaged over the levels of: Percentile 
## Degrees-of-freedom method: satterthwaite 
## Confidence level used: 0.95 
## Conf-level adjustment: scheffe method with dimensionality 2 
## P value adjustment: scheffe method with dimensionality 2
## Percentile = 0.25:
##  Condition cue_correct_fct    lsmean         SE     df   lower.CL
##  Binocular 0               0.2702125 0.05318260 464.05 0.16384729
##  CFS       0               0.1795816 0.05318260 464.05 0.07321641
##  Binocular 1               0.3093042 0.05621347 523.92 0.19687725
##  CFS       1               0.2286941 0.05663217 532.29 0.11542972
##   upper.CL t.ratio p.value
##  0.3765777   5.081  <.0001
##  0.2859468   3.377  0.0235
##  0.4217311   5.502  <.0001
##  0.3419584   4.038  0.0029
## 
## Degrees-of-freedom method: satterthwaite 
## Confidence level used: 0.95 
## Conf-level adjustment: scheffe method with dimensionality 4 
## P value adjustment: scheffe method with dimensionality 4

Extra details about mixed-model

Here are a couple of details about how I specified the mixed-effects model. I fit the model with the R package lme4 (I think we had a lab meeting dedicated to fitting mixed-effects model with this package a while ago). I assumed fixed effects for the Condition, Percentile, and naming accuracy. Percentiles were treated as a categorical (not continuous) variable. The model I fit included interactions for all of these terms. Additionally, I included a random intercept for participants. However, I did not include any random slopes (i.e., no interaction between participant and the other predictors). Finally, following Kevin’s advice, I fit the model to differences of quantiles, rather than the quantiles directly.

Fitting exponentially modified Gaussian distribution

This next section contains some preliminary results from fitting an exponentially modified gaussian distribution to the RTs. As above, I only looked at ‘go’ trials in which participants correctly responded go. The three parameters of the exgaussian were fit using a Bayesian Heirarchical framework. As with the mixed-model above, I always coded participants as random effects (i.e., each participant had their own unique intercept, and these intercept were drawn from a single hyperprior distribution which was normally distributed and whose variance was itself estimated). I then considered just a few different combinations of model parameters. The model with the most parameters that I looked at allowed the location and variance of the guassian part of the exgaussian – as well as the rate parameter of the exponential – to vary by Condition, whether the aperature was named correctly, and an interaction between the two. I fit two reduced models, one in which the Condition did not affect the mean of the exgaussian and another in which the Condition did not affect either the rate or variance parameters.

WAIC for model comparison

As a first pass to see the affect of condition, I tried a model comparison approach to see whether study condition affected either the location parameter or one of the other two parameters. This revealed that the addition of parameters to allow a study condition-dependent shift in RT meaningfully increased the fit of the model. However, allowing the rate parameter in the exponential component or the variance of the normal component to vary did not meaningfully increase the model fit.

There are many combinations of parameters that I haven’t looked at (e.g., allowing just one of either the rate or the variance to shift by condition). How many of these do you think it’s important to look at? For example, I could consider looking closer at the rate of the exponential.

Posterior Predictive Check Summaries

Here are three plots to assess the fit of the model (given the WAIC of the models, I’m only using a model where study condition affects the location of the exgaussian, but doesn’t affect the variance or rate parameters). The first plot is a histogram of the residual error of the data as compared to 12 iterations of sampling from the posterior distribution of the model. That is, the posterior distribution of parameter fits were used to generate 12 different datasets, each with the same number of observations as the original data. Each of the 12 histograms show the results of subtracting one of these simulated datasets from the original data. If the model is capturing all major trends, the histograms should be approximately normally distributed, centered at 0. If the model wasn’t capturing major trends in the data, the histograms would appear systematically non-normal or shifted (e.g., if they were centered on a value larger than 1, it would suggest that the model consistently predicted too low of RTs).

The second and third plots are much like the first, but the plots are broken into the relevant fixed effects, the interpretation is the same as the first plot, but this is just a check to make sure that all groups were being fit equally well.

Note that the third plot breaks the data into whether or not the aperture was named correctly (named == 1). When collapsed across conditions, most objects weren’t named, so those bars on the right are much smaller than those on the left.

Posterior Distribution

Here is a quick summary of the posterior distribution of the effect of study on RT (using the model where Condition only affected the location of the exgaussian). This shows the median of the posterior, with error bars extending to from the 2.5 to 97.5 quantiles of the posterior. These distributions being below 0 indicates a significant speed-up.